Biometric data capturing and analysis

ABSTRACT

A health condition of a person may be assessed from a thermal sensor signal. By increasing performance indices of a thermal camera (for example, resolution, frame rate, sensitivity), operation may be extended to identification verification, biometric data extraction and health condition analysis, and so forth. Prediction may be carried out by monitoring a time sequence of thermal images, and consequently early warning of the health condition may be provided. The apparatus may be used for, but not limited to, personalization of smart home devices through supervised and reinforcement learnings. The application of the apparatus may be, but not limited to, smart homes, smart buildings and smart vehicles, and so forth.

This patent application is a divisional application of and claimspriority to U.S. patent application Ser. No. 16/559,814 entitled“Biometric Data Capturing and Analysis” filed Sep. 4, 2019, which claimspriority to provisional patent application Ser. No. 62/730,160 entitled“Biometric Data Capturing and Analysis” filed on Sep. 12, 2018, both ofwhich are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

Aspects of the disclosure relate to extracting biometric data from asequence of thermal sensor images. The thermal sensor may comprise of anarray of thermal sensing elements to increase the performance.

BACKGROUND OF THE INVENTION

Image sensors are popular for home applications. Examples include thoseused for a baby monitor, internet protocol OP) camera, security camera,and so. Other image sensors include thermal cameras as well as an arrayof thermal sensors. Expanding the effective applications of imagesensors would enhance the popularity.

The need to expand the application of sensors (for example, thermalsensors) is underscored by an article reported in the Chicago Sun Timesin May 2019 about an Illinois man who died after suffering aheart-related event while driving and crashing his vehicle. The man wasdriving when he suffered a “heart-related event,” lost consciousness,and crashed his vehicle into a utility pole. After crashing into thepole, his car struck another vehicle. Preventive measures addressingsuch horrific events would certainly be beneficial to the generalpopulation.

SUMMARY OF THE INVENTION

An apparatus uses a thermal sensor for biometric data extraction andtracking for smart home applications. Applications such as healthcondition analysis, motion estimation (for example, fall estimation),casual prediction (for example, heart beat is slowing down to ahazardous level), hazard detection (for example, laying down for a longtime), learning the profile of individuals, and system adaptationaccording to individual preferences.

With another aspect, parameters of a thermal sensor may be enhanced toallow as much data to be extracted as possible. Examples include, butnot limited to: increasing the number of sensing element (i.e., theresolution), frame rate, sensitivity, and/or signal-to-noise level.

With another aspect, signal processing techniques extract biometric datafrom the thermal images.

With another aspect, an analytic model is used for hazard prediction andsubsequent) associated actions taken.

With another aspect, hazard analysis is done by a deep learning model.Actions are taking based on the hazard coefficients with the associatedconfidence levels estimated from the model.

With another aspect, the model would suggest actions to be taken withthe associated confidence levels based on the input data sequence.

With another aspect, the model may be trained to predict the hazardcoefficients, and the corresponding actions if necessary, with thecorresponding confidence levels based on the events previouslyoccurring.

With another aspect, the model may reside in a cloud server rather thana local processing unit for applications that are less time critical.

With another aspect, parameters of a smart device are configureddifferently based on a thermal signature of a detected person.

With another aspect, an executed application is changed from a firstapplication to a second application based on a detected conditiondetected by the first application.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary of the invention, as well as the followingdetailed description of exemplary embodiments of the invention, isbetter understood when read in conjunction with the accompanyingdrawings, which are included by way of example, and not by way oflimitation with regard to the claimed invention.

FIG. 1 shows a thermal sensor positioned in a room in accordance with anembodiment.

FIG. 2 shows an apparatus interfacing with one or more thermal sensorsand one or more associated smart devices in accordance with anembodiment.

FIG. 3 a shows an apparatus that processes information from one or morethermal sensors in accordance with an embodiment.

FIG. 3 b shows an apparatus that processes information from one or morethermal sensors in accordance with another embodiment using a deeplearning model to estimate the hazard coefficients.

FIG. 3 c shows an apparatus that processes information from one or morethermal sensors in accordance with another embodiment using a deeplearning model to suggest the actions.

FIG. 4 shows a process that identifies a user from thermal sensorinformation and applies a corresponding profile in accordance with anembodiment.

FIG. 5 shows a flowchart for executing a plurality of applications inaccordance with an embodiment.

FIG. 6 shows a flowchart for configuring a smart device with one of aplurality of parameter sets based on detected thermal signatures inaccordance with an embodiment.

FIG. 7 shows a vehicular system for continuously monitoring a vehicularoperator's physical health in accordance with an embodiment.

FIG. 8 shows a process that performs one or more actions based adetected physical condition of a vehicle driver in accordance with anembodiment.

DETAILED DESCRIPTION

According to an aspect of the embodiments, performance indices (forexample, resolution, frame rate, and sensitivity) of a thermal sensor oran array of thermal sensors may be increased to support applicationssuch as identification verification, biometric data extraction, andhealth condition analysis. Prediction may be carried out by monitoring atime sequence of thermal images and consequently an early warning of thehealth condition may be generated.

With another aspect of the embodiments, the frame rate of a thermalsensor may be increased to a determined level to capture the change inthe minor details in the thermal radiation from a human body againsttime, for example, the detail change in the thermal radiation from humanbody.

With another aspect of the embodiments, the thermal image of the bloodflows through the skin may be converted to a time signal for pulse rateextraction. Further signal processing techniques may be applied to extrabiometric data of an individual for analyzing the health condition. Animage signal may be processed to identify multiple objects from thecontent and to track associated biometric data.

With another aspect of the embodiments, an application may determine theposition of a human body within the image signal, together with motiontracking from the previous images, for fall detection. Motion estimationmay be applied to predict if there is any hazard to the individualswithin the image signal.

With another aspect of the embodiments, a profile may be associated toan individual. An apparatus may track and learn the behavior of theindividual from the history of image signal. Moreover, the apparatus mayadapt when the individual is detected in the scene. For example, the settemperature of the air conditioner in the sitting room may be adapted toan individual's preference when the individual is detected going intothe sitting room in the summer time.

With another aspect of the embodiments, the environment temperature canbe controlled according to the body temperature of individual(s),together with other parameters (such as relative humidity and outsidetemperature, and so forth) to reach the overall comfort zone throughmachine learning.

With another aspect of the embodiments, the accuracy of an analysis isdetermined by the resolution, sampling frequency and sensitivity of athermal sensor, signal processing techniques in extracting biometricdata from the image signals, and analytic/learning algorithms.

With another aspect of the embodiments, applications of thermal sensorsmay be extended to domestic applications.

With another aspect of the embodiments, the analytic model is composedof a trained model. The model is trained from a database of referencethermal image signals and an associated target vector, which mayrepresent a series of settings for the smart home devices. Reinforcementlearning may be deployed to allow the model to adapt to a new targetvector. For example, a user may change the temperature setting of a roombetween summer and winter.

With another aspect of the embodiments, no training is applied to theanalytic model but learning from the sequence of target vectors overtime that is associated with a thermal signature. For example, when anew thermal signature, which is associated with a new user, is detected,a default setting for the smart him devices is applied. When the userchanges the setting of individual device, the new setting would berecorded for re-training the model.

FIG. 1 shows thermal camera 101 positioned in room 100 in accordancewith an embodiment. Camera 101 may generate thermal image (thermogram)102 of an individual not explicitly shown.

With some embodiments, thermal camera 101 comprises a lens that focusesinfrared or far-infrared radiation by objects in view. The focused lightis scanned by a thermal sensor, which comprises a plurality ofinfrared-detector elements (for example, 24 by 32 pixels). The detectorelements may create a very detailed temperature pattern (for example,thermogram 102).

With some embodiments, camera 101 may require one-hundredth of a secondfor the detector array to obtain sensor information to obtain thethermal image. The sensor information may be periodically obtained fromseveral thousand points in the field of view of the thermal sensor toform a sequence of thermal images.

Thermogram 102 created by the detector elements of the thermal sensormay be converted into electric impulses. The impulses are then sent to asignal-processing unit (for example, apparatus 300 as shown in FIG. 3 ),which may be implemented as a circuit board with a dedicated chip thatconverts the sensor information into biometric data.

Thermal camera 101 may also include a tracking capability so that thedirection of camera 101 may vary to track a moving object such as person102 moving in room 100.

While FIG. 1 depicts one thermal sensor, some embodiments may interfacewith a plurality of thermal sensors. For example, thermal sensor arraysmay be positioned in different rooms and/or at entry points of adwelling.

FIG. 2 shows apparatus 200 interfacing with thermal sensor 204 and/or205 through sensor interface 206 and smart devices 202 and/or 203through smart device interface 209 in accordance with an embodiment.

Thermal sensors 204 and 205 are often used for access control andpresence detection. With some embodiments, in order for processor 201 toextract biometric data from sensor information, the performance ofthermal sensor 204 may be extended by increasing the sample frequency(for example, frame rate) of capturing the image signal, identifying andtracking individuals from the image signal, and analyzing detail changesin thermal images against time. Processor 201 may convert sensorinformation (signals) to biometric data, such as heart rate, bodyposition, health condition, and so forth. Apparatus 200 may also supportprediction of future health events may by processing the image signalsand/or support system personalization.

With some embodiments, processor 201 may process sensor information todetect a thermal signature of a user. When a thermal signature of aparticular individual is detected, processor 201 may apply theindividual's profile (for example, a temperature setting) to smartdevice 202 (for example, an air conditioner).

Processor 201 may support one or more health applications that processesand/or analyzes biometric data and may generate notifications about thebiometric data to an external entity (for example, a doctor) overcommunications channel 251 via interface 210. As an example, a healthapplication may detect that a user is having a possible heart attackfrom the biometric data; consequently, an urgent notification is sent tothe user's doctor about the event.

With reference to FIG. 2 , a computing system environment may include acomputing device where the processes (for example, process 300 shown inFIG. 3 ) discussed herein may be implemented. The computing device mayinclude processor 201 for controlling overall operation of the computingdevice and its associated components, including RAM, ROM, communicationsmodule, and first memory device 207. The computing device typicallyincludes a variety of computer readable media. Computer readable mediamay be any available media that may be accessed by computing device andinclude both volatile and nonvolatile media, removable and non-removablemedia. By way of example, and not limitation, computer readable mediamay comprise a combination of computer storage media and communicationmedia.

Computer storage media may include volatile and nonvolatile, removableand non-removable media implemented in any method or technology forstorage of information such as computer readable instructions, datastructures, program modules or other data. Computer storage mediainclude, but is not limited to, random access memory (RAM), read onlymemory (ROM), electronically erasable programmable read only memory(EEPROM), flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium that can be used to store the desired informationand that can be accessed by the computing device.

Communication media typically embodies computer readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. Modulated data signal is a signal thathas one or more of its characteristics set or changed in such a manneras to encode information in the signal. By way of example, and notlimitation, communication media includes wired media such as a wirednetwork or direct-wired connection, and wireless media such as acoustic,RF, infrared and other wireless media.

With some embodiments, processor 201 may execute computer-executableinstructions stored at memory 207 and access profile data stored atmemory 208.

With some embodiments, memory devices 207 and 208 may be physicallyimplemented within a single memory device.

FIG. 3 a shows apparatus 300 a that processes information from one ormore thermal sensors 301 in accordance with an embodiment.

By using a higher quality thermal sensor 301 (for example, with a framerate of at least 100 frames per second, resolution of at least 24×32pixels, good sensitivity, and low noise), biometric data 351 may beextracted via appropriate signal processing techniques via analog frontend 302, analog to digital convertor (ADC) 303, and feature extractor305. Biometric data 351 may include pulse rate, body temperature,temperature distribution pattern, body contour and posture, and soforth. By tracking the variations of biometric data, the healthcondition of an individual may be analyzed by analyzer 306, and earlywarning signals 352 and 353 may be generated by analyzer 306 and actiongenerator 307, respectively, by further processing biometric data 351.

An application may utilize a domestic thermal camera installed for falldetection by tracking the change of posture. For example, when theposture changes from upright to horizontal in a short time, a possiblefall may be detected and hence an associated alert may be generated.Moreover, a variation of posture, body temperature, temperaturedistribution pattern, and heart rate may be tracked to estimate thehazard level, and associated actions 353 can be taken.

Hazard prediction from biometric data 351 may also supported. Forexample, when one's body temperature is continuously dropping andhis/her posture is shaking, the chance of a fall may be higher (asindicated by hazard level 352) and hence an alert may be generated(triggered) before a fall can occur.

Block 302 may perform signal amplification and non-recursive band-passfiltering, in which the analog signal corresponding to the thermal imageis processed for DC offset removal, noise reduction and frequencylimited before being processed by ADC 303. (With some embodiments, block303 may comprise a 16-bit ADC with the sampling frequency (for example,200 Hz) being set high enough to capture the details of temperaturechange of an object.)

In the feature extraction block 305, image processing is applied toidentify valid objects, track the thermal profile of individual objectsover time and extract the parameters from the thermal profile to form afeature vector. Examples of the parameters for the feature vectorinclude period time, variation of the period times, certain timeconstants within each periodic cycle and their variations over time,etc. The analytic model 306 takes in the feature vector and compares itover a trained model. The model is pre-trained with a large set ofgeneric feature vectors using deep learning algorithms, e.g. a deepneural network. Reinforcement learning may be deployed to allow themodel to learn from the mistakes. Hazard levels may be provided for theidentified objects. In block 307, a list of actions may be pre-definedand may be triggered based on the associated hazard levels.

FIG. 3 b shows apparatus 300 b that processes information from one ormore thermal sensors 301 in accordance with another embodiment. Hazardcoefficients 322 with associated confidence levels 321 are estimated bya model trained using a deep learning model 308, for example, aconvolutional neural network with supervised learning. Actions 324 aredetermined from action list 323 and may be based on hazard coefficients322 and confidence level 321 provided by model 308. Model 308 mayinitially support hazard levels but subsequently identify differenthazards with more empirical data such as heart-rate abnormality, bodytemperature drop, fall detection, and so forth.

Model 308 in apparatus 300 b may also be trained to predict hazards,rather than estimating hazards, based on the training sequence whichstarted from a substantially earlier time.

FIG. 3 c shows apparatus 300 c that processes information from one ormore thermal sensors 301 in accordance with a third embodiment, in whichactions 332 and associated confidence levels 331 are estimated bytrained model 309. Again, model 309 in apparatus 300 c also may betrained to predict any actions needed.

The image processing technique that may be used depends on the systemcomplexity, including the number of thermal sensors, the resolution ofeach thermal sensor, the list of hazards and actions, the systemcomputation power and memory available, and so forth.

For the embodiments shown in FIGS. 3 a, 3 b and 3 c , the analyticmodels may be implemented locally or in a cloud server, depending oncriticality of response time.

FIG. 4 shows process 400 that identifies a user from thermal sensorinformation and applies a corresponding profile in accordance with anembodiment.

Process (application) 400 supports human presence detection and thermalsignature verification at block 401. If a human object is detected andthe thermal signature is matched to a known entity at block 401, allsupported smart devices (for example, air conditioner, smart TV, orsmart lighting) may be adjusted 403 in accordance with the profiledatabase stored at block 402.

If there is any adjustment to the applied profile 405, the adjustmentdata 453 may be sent to a profile adaptation unit 406 in which the newsettings in the profile may be included. The profile database would beupdated 451 by profile adaptation unit 406 if an adjustment is needed.

To add a new user, profile adaptation unit 406 sends the thermalsignature of the new user 453 to the user identifier unit 401 togetherwith the associated profile, which could be a default profile, to theprofile database unit 402.

Profile adaptation unit 406 may comprise a deep learning model trainedusing reinforcement learning.

FIG. 5 shows flowchart 500 for sequencing through a plurality ofapplications, as executed by apparatus 200, in accordance with anembodiment. Apparatus 200 may execute one of the plurality ofapplications depending on a detected condition. For example, a firsthealth application may monitor general health measurements (for example,an amount of activity and temperature) of a user. If one or more of themeasurements are abnormal, apparatus 200 may initiate different healthapplications based on the detected condition.

Referring to FIG. 5 , apparatus 200 configures thermal sensors 204 and205 in accordance with a first set of sensor parameters at block 501 inorder to execute a first application at block 502.

If an abnormal condition is detected at block 503, apparatus 200initiates an appropriate application. For example, apparatus maytransition to a second application to monitor fall prediction or to athird application to monitor the heart rate of the user at blocks504-505 and 506-507, respectively. When executing the second or thirdapplications, apparatus 200 may configure thermal sensors 204 and 205differently in order to obtain different biometric data.

In another implementation, different configuration parameters may beapplied to individual sensor for each application.

In a third implementation, different sets of configuration parametersare applied to the sensors one after another to extract all thebiometric data before running the applications.

In a fourth implementation, a most comprehensive set of configurationparameters is used for all sensors and applications. All of the sensorsmay be set to the best set of configuration, for example but no limitedto, highest image resolution, number of bits, frame rate, sensitivity,signal to noise ratio (SNR), computational power, power consumption, andso forth.

FIG. 6 shows flowchart 600 where apparatus 200 configures a smart devicewith one of a plurality of parameter sets based on detected users inaccordance with an embodiment. Apparatus 200 may monitor sensor datafrom thermal sensors 204 and/or 205 to detect thermal signatures of oneor more users. For example, thermal sensor 204 may be positioned at anentry point of a dwelling. Based on sensor information obtained fromsensor 204, apparatus 200 may identify users entering and exiting thedwelling. With some embodiments, apparatus may detect either a thermalsignature from the front (corresponding to a person entering thedwelling) or from the back (corresponding to the person exiting thedwelling). Based on the detected thermal signatures, a smart device canbe configured with different sets of parameters (for example, thetemperature setting of an air conditioner).

At block 601, apparatus 200 trains to detect thermal signatures ofdifferent users from sensor data. For example, distinguishingcharacteristics may be stored at memory 208. When thermal signatures ofboth users are detected at block 602, only user A at block 604, or onlyuser B at block 606, a smart device may be configured in accordance witha first set of smart device parameters at block 603, a second set atblock 605, or a third set at block 607, respectively. With someembodiments, the first set (when both users are detected) may be acompromise between the second and third sets (when only one user isdetected). Otherwise (when no users are detected), the smart device maybe configured in accordance with a default set of smart deviceparameters at block 608.

The following capabilities may be supported by the embodiments.

An apparatus uses a thermal sensor for biometric data extraction andtracking for smart home applications. Applications such as healthcondition analysis, motion estimation (for example, fall estimation),casual prediction (for example, heart beat is slowing down to hazardlevel), hazard detection (for example, laying down for a long time),learning the profile of individuals, and system adaptation according toindividual preferences.

The parameters of a thermal sensor may be enhanced to allow as much datato be extracted as possible. Examples include, but not limited to:

-   -   a. Increasing the resolution, frame rate, sensitivity and        signal-to-noise level, for example, for heart rate monitoring.    -   b. Increasing the resolution, sensitivity and signal-to-noise        level, and so forth for detection distance.    -   c. Increasing the resolution for the number of tracked objects.

Signal processing techniques extract biometric data from thermal images.

Analytic model for hazards estimation and subsequently the associatedactions taken.

Analytic model for actions estimation.

Analytic model for hazards and/or actions prediction.

Model for learning the behaviors of individual(s) to the smart devicesaccording to the biometric data extracted from the thermal sensors.

Configure parameters of a smart device based different detected people.

Change to a second health application from a first health applicationbased on a detected condition by the first health application. The setof configuration parameters for individual sensors for an active healthapplication may or may not be identical.

Use different set of configuration parameters to extract all biometricdata before running the health applications.

Using a single comprehensive set of configuration parameters for all thesensors and health applications.

Obtain thermal sensor data to detect a thermal signature for either thefront or the back of a person.

Able to increase the sampling frequency of thermal sensors, including IPcameras, thermal cameras, and thermal sensors, to capture the minorchanges of the color content due to thermal radiation from a human body.

Able to increase the resolution and sensitivity of thermal sensors tospan the detection range.

Exemplary Clauses:

1. An apparatus supporting at least one smart device, the apparatuscomprising:

-   -   a smart device interface;    -   a thermal sensor interface configured to obtain sensor        information from a first thermal sensor;    -   a processor for executing computer-executable instructions;    -   a memory storing the computer-executable instructions that when        executed by the processor cause the apparatus to perform:        -   detecting, from the sensor information, a detected thermal            signature of a detected user;        -   when the detected user is a first user, obtaining a first            profile corresponding to the first user, wherein the first            profile comprises a first set of smart device parameters;            and        -   when the detected user is the first user, configuring,            through the smart device interface, a first smart device            based on the first set of smart device parameters.            2. The apparatus of clause 1, wherein the memory storing            computer-executable instructions that when executed by the            processor further cause the apparatus to perform:    -   when the detected user is a second user, obtaining a second        profile corresponding to the second user, wherein the second        profile comprises a second set of smart device parameters and        wherein the second set is different from the first set; and    -   when the detected user is the second user, configuring, through        the smart device interface, the first smart device based on the        second set of smart device parameters.        3. An apparatus supporting at least one smart application, the        apparatus comprising:    -   a thermal sensor interface configured to obtain sensor        information from a first thermal sensor;    -   a processor for executing computer-executable instructions;    -   a memory storing the computer-executable instructions that when        executed by the processor cause the apparatus to perform:    -   when executing a first application:        -   configuring the thermal sensor in accordance with a first            set of sensor parameters;        -   when the thermal sensor is configured with the first set of            parameters, extracting biometric data from the sensor            information; and        -   when a first condition is detected from the biometric data,            initiating a second application; and    -   when executing the second application,        -   configuring the thermal sensor in accordance with a second            set of parameters, wherein the first and second sets differ            by at least one parameter; and        -   when the thermal sensor is configured with the second set of            sensor parameters, extracting the biometric data from the            sensor information.            4. An apparatus supporting at least one smart application,            the apparatus comprising:    -   a thermal sensor interface configured to obtain sensor        information from a thermal sensor and configuring the thermal        sensor in accordance with a most comprehensive set of sensor        parameters for all applications;    -   a processor for executing computer-executable instructions;    -   a memory storing the computer-executable instructions that when        executed by the processor cause the apparatus to perform:    -   extracting biometric data from the sensor information;    -   executing the first application;    -   executing the second application.        5. An apparatus supporting at least one smart application with        more than one thermal sensors, the apparatus comprising:    -   a first thermal sensor interface configured to obtain sensor        information from a first thermal sensor;    -   a second sensor interface configured to obtain sensor        information from a second thermal sensor;    -   a processor for executing computer-executable instructions;    -   a memory storing the computer-executable instructions that when        executed by the processor cause the apparatus to perform:    -   configuring the first thermal sensor in accordance with the        first set of sensor parameters;    -   configuring the second thermal sensor in accordance with the        second set of sensor parameters;    -   extracting biometric data from the sensor information from all        the sensors;    -   executing the first application:    -   executing the second application.    -   With some embodiments, the sets of configuration parameters for        all the sensors may be identical, in other words, all sensors        can be configured with a most comprehensive set of parameters        for all applications. The best sensor configuration may include,        but not limited to, highest image resolution, number of bits,        frame rate, sensitivity, and signal to noise ratio (SNR).

The following is directed to vehicle operator continuous physical healthmonitoring embodiments.

Referring back to FIG. 1 , while embodiments support assessing thehealth of a person in a room using a thermal sensor, embodiments mayutilize thermal sensor data to assess the health of a person withinother types of confined spaces such as a vehicle. The parameters usedmay include, but is not limited to, heart rate, breathing rate, bodytemperature, posture (in particular, head position), and thetrajectories of these data over time, and so forth.

The physical health condition of a vehicle operator (vehicle driver) maybe critical to the safety of the operator, the passengers, and thevehicle itself. The state of the vehicle operator condition coulddetermine the output of a situation should an emergency arisesunexpectedly.

With traditional approaches, there are numbers of ways to monitor thephysical health of the vehicle operator via wearables devices. However,a wearable device is specific to the individual wearing the device andnot to the vehicle and may not ensure that the information or data ofthe vehicle operator's health is securely monitored during the durationof the vehicle when it is in use.

With an aspect of the embodiments, monitoring of a driver and/or vehiclemay be performed in an non-intrusive and accurate manner that isactivated all of the time that the vehicle is in operation.Consequently, the health of whoever is driving the vehicle may beassessed. With this approach, biometric information about the driver isutilized for accident prevention, incident alert, critical healthwarning and postmortem analysis.

FIG. 7 shows vehicular system 700 for continuously monitoring avehicular operator's physical health in accordance with an embodiment.

In reference to FIG. 2 , the embodiment obtain thermal sensor data fromthermal sensor 204 via thermal sensor interface 206 as previouslydiscussed.

The thermal sensor 204 is typically fitted at a fixed location in frontof the vehicle operator (driver), for example, mounted against the topwindshield corner in front of the driver.

Processor 703 configures the thermal sensor by reference to methods inFIG. 5 .

Processor 703 extracts biometric information contained in sensor data750. For example, processor 703 may continuously monitor the heart rateand head posture about the driver as soon as he sits in the drivingseat. In addition, the health record of the driver may be loaded intoprocessor 703 via wireless devices 704 from a remote database server.

Processor 703 may decide addition biometric data are needed based on thehealth record of the driver. For example, if the BMI of the driverexceeds a certain value, the change of heart rate, the change of bodytemperature, and change of head posture over time may also be monitored.

As will be discussed in further detail, processor 703 detects one ormore current physical conditions about the driver and executes one ormore actions to address the detected physical conditions.

Processor 703 may report detected physical conditions to the driver,doctor, emergency contact, and so forth via wireless device 704 (forexample a smartphone) executing an application, initiating a telephonecall to 911, generating an e-mail message to a designated person, and soforth.

Processor 703 may also initiate an action in response to the detectedphysical condition. For example, if processor 703 determines that thedriver is experiencing a heart attack, processor may instructself-driving interface 704 to route the vehicle to the nearest hospital.

As will be further discussed, biometric information may be stored instorage device 706 for subsequent analysis about the health condition ofthe vehicle driver. While storage device 706 is shown as a separatedevice, storage device 706 may be integrated within wireless device 704.

FIG. 8 shows process 800 that performs one or more actions based adetected physical condition of a vehicle driver in accordance with anembodiment.

At block 801, processor 703 extracts biometric information contained insensor data 750. Processor 703 processes the information conveyed insignal 750 to extract measurements for one or more biometriccharacteristics of the vehicle driver at block 802. Biometriccharacteristics may include, but are not limited to, heart rate,breathing rate, and deviation from average heart rate (for example,degree of heart beat irregularity).

The measurements of the biometric characteristics may be stored instorage device 706 for analysis about the health condition of thevehicle driver at a later time. For example, the stored data may beevaluated by the driver's doctor to determine if medical treatment isneeded.

At block 803, process 800 obtains the measurements of the biometriccharacteristics (for example, the vehicle driver's heart rate andbreathing rate) and determines whether a health profile applies to thedrives. A plurality of health profiles may be specified, where a firsthealth profile maps to normal vital functions of the driver (in otherwords, no detected health event), a second health profile maps to aheart attack event, a health third profile maps to the driver fallingasleep, a fourth health profile maps to excessive alcohol consumption,and so forth.

If an abnormal health is detected based on the determined health profileis detected at block 804, process 800 detects whether a particularhealth event occurred at blocks 805-809. Based on a particular healthevent, process 800 executes an appropriate action. Exemplary actionsinclude, but are not limited to:

-   -   Sleep event (block 805—driver falling asleep): initiate a loud        warning sound through the vehicle radio or wireless device to        alert the driver    -   Heart attack event (block 806): instruct a self-driving        interface to drive the vehicle to the nearest hospital    -   Excessive alcohol consumption (block 807): prevent the vehicle        driver from starting the car or safely parking the car if the        car is moving    -   Arrhythmia event (block 808—irregular heart beat or missing        heart beats): generating an alert to the driver through a        wireless device    -   Stroke event (block 809): instruct a self-driving interface to        drive the vehicle to the nearest hospital

With an aspect of the embodiments, a processing unit continuouslymonitors and analyzes the heartbeat of a vehicle driver to generate analert about any irregularity. The processing unit may use a uniquealgorithm to provide this capability.

With an aspect of the embodiments, a processing unit may identifying adetected irregularity to correspond to one of a plurality of eventsabout a vehicle driver, including, but not limited to, falling asleep, aheart attack, consuming an excessive amount of alcohol, and so forth.

With an aspect of the embodiments, data about the heartbeat of a vehicledriver may be stored in a storage device. The data may be retrieved at alater time for analyzing whether an abnormal health event occurred.

As can be appreciated by one skilled in the art, a computer system withan associated computer-readable medium containing instructions forcontrolling the computer system can be utilized to implement theexemplary embodiments that are disclosed herein. The computer system mayinclude at least one computer such as a microprocessor, digital signalprocessor, and associated peripheral electronic circuitry.

What is claimed is:
 1. A method for assessing a health condition of avehicle driver, the method comprising: generating, by a thermal sensor,a continuous image signal that is reflected by the vehicle driver;extracting, by a processing unit from the continuous image signal, afirst measurement for a first biometric characteristic of the vehicledriver; mapping the first measurement to a first health profile of aplurality of health profiles, wherein the plurality of health profilescomprises at least two specific illnesses for the vehicle driver; anddetecting whether the first health profile is indicative of a firstabnormal health event.
 2. A method for assessing a health condition of avehicle driver, the method comprising: generating, by a thermal sensor,a continuous image signal that is reflected by the vehicle driver;extracting, by a processing unit from the continuous image signal, afirst measurement for a first biometric characteristic of the vehicledriver; mapping the first measurement to a first health profile of aplurality of health profiles; detecting whether the first health profileis indicative of a first abnormal health event; identifying the vehicledriver; downloading health record of the identified vehicle driver;determining, based on the downloaded health record, whether additionalbiometric data about the vehicle driver is required; and in response tothe determining, monitoring the additional biometric data.
 3. The methodof claim 2, further comprising: monitoring a heartbeat characteristic ofthe vehicle driver.
 4. The method of claim 3, further comprising:detecting an irregularity from the heartbeat characteristic of thevehicle driver; and in response to the detecting, generating an alertabout the irregularity.
 5. The method of claim 4, further comprising:distinguishing a type of irregularity from a plurality ofirregularities.
 6. The method of claim 3, further comprising: storingdata about the heartbeat characteristic.
 7. The method of claim 2,further comprising: when the first abnormal health event is detected,initiating, by the processing unit, a first action.
 8. The method ofclaim 2, further comprising: extracting, by the processing unit, asecond measurement for a second biometric characteristic of the vehicledriver; mapping the first measurement and the second measurement to asecond health profile of the plurality of health profiles; detectingwhether the second health profile is indicative of a second abnormalhealth event; and when the second abnormal health event is detected,initiating, by the processing unit, a second action.
 9. The method ofclaim 2, further comprising: obtaining, from the thermal sensor, athermal sensor signal; and extracting, by the processing unit, biometricdata from the thermal sensor signal.
 10. The method of claim 2, whereinthe monitoring comprises: downloading a downloaded thermal signature ofthe identified vehicle driver; obtaining, from the thermal sensor, adetected thermal signature of the vehicle driver; comparing thedownloaded thermal signature with the detected thermal signature; and inresponse to the comparing, matching the detected thermal signature to aknown entity.
 11. The method of claim 2, wherein the identifyingcomprises: authenticating the vehicle driver before downloading thehealth record of vehicle driver; and when the authenticating issuccessful, enabling the downloading of the health record of theauthenticated vehicle driver.
 12. The method of claim 2, wherein thedetermining comprises: when a body mass index (BMI) of the vehicledriver exceeds a predetermined amount, selecting one of a plurality ofbiometric characteristics, wherein said one is different from the firstbiometric characteristic; and measuring said one of the plurality ofbiometric characteristics.
 13. An apparatus providing an assessment of avehicle driver of a vehicle, the apparatus comprising: a thermal sensorconfigured to receive continuous image sequence reflected by the vehicledriver; a processing unit for executing computer-executableinstructions; a memory storing the computer-executable instructions thatwhen executed by the processing unit cause the apparatus to perform:extracting, by the processing unit from an image signal, a firstmeasurement for a first biometric characteristic of the vehicle driver;mapping the first measurement to a first health profile of a pluralityof health profiles; detecting whether the first health profile isindicative of a first abnormal health event; when the first abnormalhealth event is detected, initiating, by the processing unit, a firstaction; identifying the vehicle driver; downloading health record andthermal signature of the identified vehicle driver; determining, basedon the downloaded health record, whether additional biometric data aboutthe vehicle driver is required; and in response to the determining,monitoring the additional biometric data.
 14. The apparatus of claim 13further comprising: a self-driving interface; and the memory storing thecomputer-executable instructions that when executed by the processingunit further cause the apparatus to perform: instructing, through theself-driving interface, the vehicle to drive to an appropriatedestination.
 15. The apparatus of claim 13, wherein the memory storingthe computer-executable instructions that when executed by theprocessing unit further cause the apparatus to perform: extracting, bythe processing unit, a second measurement for a second biometriccharacteristic of the vehicle driver; mapping the first measurement andthe second measurement to a second health profile of the plurality ofhealth profiles; detecting whether the second health profile isindicative of a second abnormal health event; and when the secondabnormal health event is detected, initiating, by the processing unit, asecond action.
 16. One or more non-transitory computer-readable mediastoring instructions that, when executed by a computing systemcomprising at least one processor, a thermal sensor, and memory, causethe computing system to: obtain, from the thermal sensor, a continuousimage signal that is reflected by a vehicle driver; extract, from thecontinuous image signal, a first measurement for a first biometriccharacteristic of the vehicle driver; map the first measurement to afirst health profile of a plurality of health profiles; detect whetherthe first health profile is indicative of a first abnormal health event;identify the vehicle driver; download health record of the identifiedvehicle driver; determine, based on the downloaded health record,whether additional biometric data about the vehicle driver is required;and in response to the determining, monitor the additional biometricdata.